State-similarity metrics for continuous Markov decision processes

In recent years, various metrics have been developed for measuring the similarity of states in probabilistic transition systems (Desharnais et al., 1999; van Breugel & Worrell, 2001a). In the context of Markov decision processes, we have devised metrics providing a robust quantitative analogue of bisimulation. Most importantly, the metric distances can be used to bound the differences in the optimal value function that is integral to reinforcement learning (Ferns et al. 2004; 2005). More recently, we have discovered an efficient algorithm to calculate distances in the case of finite systems (Ferns et al., 2006). In this thesis, we seek to properly extend state-similarity metrics to Markov decision processes with continuous state spaces both in theory and in practice. In particular, we provide the first distance-estimation scheme for metrics based on bisimulation for continuous probabilistic transition systems. Our work, based on statistical sampling and infinite dimensional linear programming, is a crucial first step in real-world planning; many practical problems are continuous in nature, e.g. robot navigation, and often a parametric model or crude finite approximation does not suffice. State-similarity metrics allow us to reason about the quality of replacing one model with another. In practice, they can be used directly to aggregate states

Computing Distances between Probabilistic Automata

We present relaxed notions of simulation and bisimulation on Probabilistic Automata (PA), that allow some error epsilon. When epsilon is zero we retrieve the usual notions of bisimulation and simulation on PAs. We give logical characterisations of these notions by choosing suitable logics which differ from the elementary ones, L with negation and L without negation, by the modal operator. Using flow networks, we show how to compute the relations in PTIME. This allows the definition of an efficiently computable non-discounted distance between the states of a PA. A natural modification of this distance is introduced, to obtain a discounted distance, which weakens the influence of long term transitions. We compare our notions of distance to others previously defined and illustrate our approach on various examples. We also show that our distance is not expansive with respect to process algebra operators. Although L without negation is a suitable logic to characterise epsilon-(bi)simulation on deterministic PAs, it is not for general PAs; interestingly, we prove that it does characterise weaker notions, called a priori epsilon-(bi)simulation, which we prove to be NP-difficult to decide.Comment: In Proceedings QAPL 2011, arXiv:1107.074

Архетип свобода у контексті французької політичної теорії та історії

Розглянуто сучасні підходи щодо аналізу політичної ментальності. У межах політологічного аналізу окреслено коло проблем, які потребують вирішення з використанням підходів психології. Зроблено висновок про те, що архетип “свобода” становить важливий елемент політичної ментальності французів.Modern approaches of analysis of political mentality are considered. Within the limits of political science analysis outlined circle of problems which need decision with the use of approaches of psychology. A conclusion is done that archetype freedom makes the important element of political mentality of French’s

Bird life of coasts and estuaries

Bird Life of Coasts and Estuaries describes the bird life of the British coastline and adjacent off-shore waters from an ecological point of view, using information from research to show how bird distribution and abundance are related to important environmental variables such as marine currents, weather, coastal landform and the influence of man. First published in 1992, the book will appeal to the layman who wants to know more about coastal birds, the birder who wants to find out how birds interact with their environment and all those who are interested in the habitats that make up what is arguably Britain's most important natural asset

Bisimulation Metrics are Optimal Value Functions

International audienceBisimulation is a notion of behavioural equivalence on the statesof a transition system. Its definition has been extended to Markov decisionprocesses, where it can be used to aggregate states. A bisimulation metricis a quantitative analog of bisimulation that measures how similar statesare from a the perspective of long-term behavior. Bisimulation metrics havebeen used to establish approximation bounds for state aggregation and otherforms of value function approximation. In this paper, we prove that abisimulation metric defined on the state space of a Markov decision processis the optimal value function of an optimal coupling of two copies of theoriginal model. We prove the result in the general case of continuous statespaces. This result has important implications in understanding thecomplexity of computing such metrics, and opens up the possibility of moreefficient computational methods

Metrics for Markov decision processes

We present a class of metrics, defined on the state space of a finite Markov decision process (MDP), each of which is sound with respect to stochastic bisimulation, a notion of MDP state equivalence derived from the theory of concurrent processes. Such metrics are based on similar metrics developed in the context of labelled Markov processes, and like those, are suitable for state space aggregation. Furthermore, we restrict our attention to a subset of this class that is appropriate for certain reinforcement learning (RL) tasks, specifically, infinite horizon tasks with an expected total discounted reward optimality criterion. Given such an RL metric, we provide bounds relating it to the optimal value function of the original MDP as well as to the value function of the aggregate MDP. Finally, we present an algorithm for calculating such a metric up to a prescribed degree of accuracy and some empirical results

Ordinal species richness in insects-a preliminary study of the influence of morphology, life history, and ecology

In an effort to understand why some groups of insects, especially beetles, are much more diverse than others, analyses were carried out on the role of morphological, life-history, and ecological factors on the species richness of various orders, as quantified by the number of extant species they each contain. The influence of body size, the form of the wings and thorax, the mode of development, parasitism, the ability to accumulate resources for egg laying at both juvenile and adult stages (ovigenic flexibility), diet range, and habitat choice were examined using comparative methods that control for phylogeny. Hierarchical multiple regression revealed that about 10% of the variance in insect species richness occurred within subclasses, 34% within superorders, and 54% within or below ordinal level. The evolution of wings and mode of development were key differences between the subclasses, and two superorders, each containing a single species-rich order and several species-poor ones, were responsible for most of the variation at superordinal level. After controlling for phylogeny, and on the basis of minimal adequate statistical models, the rate of cladogenesis declined with time, and was higher in orders with a broad range of larval diets. The number of species in each order was most closely correlated with larval diet breadth, the proportion of species able to fly, physical protection of the hindwings by the forewings at rest, the proportion of species that are parasitic, and the age of the order. Evidence from the literature on adult diet and ovigeny index, supported by measurements of the width of the thoracico-abdominal junction, indicates that the food processing ability of adult Diptera, Lepidoptera, and Hymenoptera is subject to morphological constraints limiting the contribution that the adult diet makes to the accumulation of the protein reserves needed for egg formation. Adult Coleoptera are free of such constraints, and thus exhibit the broadest range of both larval and adult diets and inhabit the widest range of habitats. Beetles can thus accumulate resources for egg production as larvae, adults, or both, as a consequence of their unique adult morphology. The key adult features involved are (1) the protective elytra and thickened exoskeleton, (2) the flexible jointing of the prothorax with the pterothorax, (3) the wide, rigid post-thoracic junction with the abdomen allowing enlargement of the gut for the storage, passage, and efficient digestion of a wide range of particulate foods, and (4) the biting mouthparts with which to process them